Free Markets and the Dangers of Optimisation Engines

In my last post, I discussed why I think that Capitalism can be a net positive for the world, along with why it shouldn’t be used as a bogeyman that stands synonymous with all of the ills and excesses of modern society.  Some of those ideas may come across as echoing the thoughts and words of Milton Friedman, who was famous for his laissez-faire free-market economics.  Friedman made some very good arguments, however I cannot agree with his entire worldview.

The issue that I have with Milton Friedman, and the free-market capitalist policies that he championed is not with the fundamental validity of the ideas.  It is well established and demonstrable that investment is generally beneficial and free-markets generally allow for higher growth and lower prices.  The problem is instead with the extremes that he took these underlying principles to.  His philosophy was one of unyielding faith that the free market could do no wrong, and that any and all attempts to regulate the market are harmful.

Friedman’s recommendations were ultimately used as justification for much of the economic policy of Thatcher and Reagan, which are still highly controversial.  I would not consider myself to be anti-free-market, but I intend to present a counterweight to the dogmatic approach of Friedman.  Both capitalism and free markets can be hugely beneficial up to a point, but it is dangerous to pursue them to the exclusion of all other considerations.

Evolution Doesn’t Care

In order to illustrate the point I am trying to make, I shall take a diversion through a completely different topic: evolution.  Our understanding of evolution has progressed significantly over the past few decades, and it is now a fairly well understood example of an optimisation engine.

Evolution optimises for survival and propagation – if something is better able to survive, or able to propagate faster, it will outcompete other things.  There is a high degree of randomness in the environment – sometimes beneficial mutations die out due to bad luck, but if a beneficial mutation is likely enough, eventually it will overcome this variation and become dominant.  Evolution however does not specifically optimise for traits that we as humans may value, such as happiness, compassion or intelligence.  If compassion or intelligence helps an individual survive and propagate, an organism with that trait will be more likely to survive, and this trait will become more prevalent.  These qualities are not guaranteed to be beneficial from an evolutionary perspective though.  Evolution doesn’t care about what humans value. (For example, Cowpea Seed Beetles (Callosobruchus maculatus) engage in copulatory wounding – a behaviour that is truly horrifying from a human perspective).

Intelligence is useful for developing better tools, and this is helpful to the intelligent individual, however if anyone that sees the tool can copy it, the additional intelligence that allows them to develop improvements confers little lasting evolutionary benefit to the individual.  There are multiple theories that attempt to explain why humans have evolved to be so much more intelligent than other animals, but none of them rely on any sort of idea that intelligence is just some fundamentally good thing that evolution is optimising for.

Several of these theories rely on the idea that intelligence allows us to model and predict other humans’ reactions to situations.  One theory is rather Machiavellian, in that our intelligence is the result of a runaway process in which humans would be more successful the more they were able to outsmart each other, leading to an intelligence arms race.  Another theory takes a slightly less misanthropic view – that collaboration is hugely beneficial to individuals within a society, provided there are no defectors, so the ability to reliably predict human behaviour allows better identification of defectors that would otherwise harm the group.  Either way, these theories suggest mechanisms in which a self-reinforcing cycle of increasing intelligence might be maintained.

These kinds of runaway processes can yield “useful” results (intelligence) or “pointless” results (peacocks’ tails).  Useful and pointless however are just human judgements – evolution “doesn’t care” about such things – it is simply mindlessly optimising for survival and propagation.

The Perpetual Arms-race

Obviously in the position Homo Sapiens is in now, we have an unprecedented capability to propagate (e.g. by leaving the planet), but evolution does not plan.  The eventual ability of space travel is entirely coincidental and did not confer an advantage to Homo Sapiens before it was fully manifested.  A different runaway evolutionary process – an arms-race between organisms developing toxins and anti-toxins would perhaps result in an extremely hostile world (consider the Amazon rainforest or perhaps the fictional xenomorphs in Alien).

Such arms races are still happening.  Just as dinosaurs were the dominant form of life before being wiped out, Homo Sapiens could be wiped out too – evolution doesn’t care.  Seeing the current state of the world with us on top, and assuming that we will stay that way is pure selection bias.

Humanity could yet be wiped out by flesh eating slugs or by a particularly virulent virus (for all the disruption that the Coronavirus pandemic is causing, things really could be so much worse).  For all we know, in another hundred million years, the planet could be dominated by the descendants of cockroaches.  These insects may or may not evolve intelligence, but human-level intelligence is only likely if there is an immediate fitness advantage to them that leads to a similar kind of arms-race scenario.

It is however exactly this kind of selection bias that makes people think that evolution is their friend, not some faceless uncaring optimisation engine.  They see the world and think “humans will continue to evolve, getting better, stronger and smarter”.  The flesh-eating slugs of the future wouldn’t care about an irrelevant species that died out millions of years ago, just as we don’t care about velociraptors today.  If we want to stay on top, we need to be aware of what is being optimised for, and find a way to satisfy that, staying on top whilst also satisfying our own desires of getting smarter, stronger, happier or whatever.

Treading this knife-edge between being a single-minded Machiavellian survivalist and a scholarly altruist is the great challenge of life.  The choice between sacrificing everything to the goal of propagating yourself, versus the desire to be a compassionate, artistic intellectual despite the likely outcome that you will be out-competed and eradicated by more ruthless players is a true conundrum.  If we sacrifice everything to propagate, what are we really propagating?  Surely there are some things that we have decided that we hold dear, that we are trying to propagate along with ourselves (like compassion, art or intelligence).  Equally though, if we hold too many things too dear – sacred cows that we cannot allow to be compromised; we risk being too inflexible – dying out, and having these ideas or qualities die with us.

When fighting disease, the optimisation engine works against us – antibiotics are great, but if used incorrectly some bacteria may survive, and the survivors will be the ones most resistant to the antibiotics.  They will then pass on their resistance, making the antibiotics less effective in future.  The way to fight disease is to act quickly enough to not give the optimisation engine time to work – hitting bacteria with a sustained barrage of antibiotics, so that a slight resistance doesn’t help them, and can’t be selected for.

There are equivalent problems with the recent proposed approaches to tackling malaria.  The female Anopheles mosquito (the primary vector for malaria) mates only once, so by releasing sterile male mosquitoes into the environment by the billion, many females will mate, have no offspring and die.  This seems like a fool proof plan to dramatically cut their population, and eventually wipe them out.  Thinking about it for a moment longer however, it becomes clear that if this plan is not executed globally in a sustained manner, wiping them out within a handful of generations, it will simply select for those rare female mosquitoes that mate more than once.  This strategy if executed imperfectly, will suppress their population for a time, then when we get bored, their population will boom, but filled with individuals on which this tactic will not work a second time – just like antibiotic resistance.

Optimisation engines are faceless and unrelenting – when they are optimising for something we want, that is good, when they are optimising for something we don’t, they have the potential to be very bad.  This is very much like the AI thought experiment about the dangers of Paperclip Maximisers, which seem to be doing a great job until they take over the world and turn it into paperclips.

Unfortunately we can’t simply “opt-out” of evolution and we ignore the reality of what is being optimised for at our peril.  We must make enough concessions to succeed without losing what we value in the process.  Getting ahead of the curve, and somehow diverting the unstoppable optimisation engine from going too far down a path we consider bad.

The Free Market

It is through this lens that we can view the free market: it is an optimisation engine.  This is an incredibly powerful force that often does exactly what we want – people freely trading with each other are able to exploit their comparative advantages to get more of what they want for less effort than they could have made it themselves.  These mutually beneficial trades increase everyone’s happiness!

Again though, we must be careful about exactly what is being optimised for: just as evolution is not optimising for intelligence, the free market is not optimising for happiness.  It is arguably optimising for profitability, and this does often coincide with progress, peace and happiness.  Indeed, if kept constrained within certain bounds, the free market is very effective at reducing costs and driving innovation.  The issue is that profitability is not completely synonymous with happiness, and happiness is important.  In optimising for profit, the following scenarios can occur:

• Marketing can convince people that their lives are miserable and inadequate, so that they need to buy things to make themselves feel better.  Without the marketing, people would be happier, but profitability is instead being maximised, so happiness gets thrown under the bus.
• Price gouging allows people or companies to profit from people in difficult or life-threatening situations, and whilst a certain amount of profit can encourage more players to enter a market (for example, getting food into a warzone is expensive and risky, so why bother if it isn’t going to be profitable), people that are sufficiently desperate are willing to take incredibly bad deals as they are unable to shop around.  This high profit environment is however a great opportunity for anyone that can corner it, which means that it would be in their interest to perpetuate the problem in order to make enormous profits for longer.
• Consolidation due to economies of scale can lead to oligopolies or monopolies, which are then able to extract supernormal returns due to the lack of competitors pushing their prices down.  Once in such a position, they are also often powerful enough to introduce barriers to entry that discourage anyone from trying to compete.  This is likely to be a natural attractor state for this kind of optimisation engine, so a completely unregulated “free market” that makes no attempt to avoid monopolies is highly likely to cease to be free, as it is taken over by monopolies.
• The drive for economic efficiency has led to an enormous amount of offshoring and specialisation – it is now evident that supply chains for many critical products are highly dependent on Chinese manufacturing.  This has led us to a geopolitical situation in which it is very difficult to challenge a country that is committing human rights abuses.  Repatriating the supply chains could take years, and without doing so, the costs of trying to sanction such a country are potentially ruinous.

When profit and human flourishing are not in alignment, we need a balance between ruthlessly optimising, and achieving our actual goals.  This can be government regulation, for example anti-monopoly legislation, marketing guidelines, environmental protections, etc.  As with evolution however, we cannot revolt without regard for what is being optimised for, or we will fail.

Mutual Existential Threat

Introducing government regulations that restrict companies in an effort to keep the market free and profit aligned with happiness sounds great.  Unfortunately though, the government doesn’t exist completely outside the system.  Any company that could profit from the removal of these regulations now has an incentive to disrupt the government.  Regulatory capture is a good example of this, in which companies use whatever tools and influence they have at their disposal to get someone sympathetic to their position installed in the relevant agency (e.g. polluters getting climate change deniers into an environmental agency).

Companies are therefore an existential threat to the government, as it is in their interest to try to capture the government and use it to their benefit.  Equally, through regulating the activities of companies and therefore reducing their potential profit, the government becomes an existential threat to those companies.  There is no way for the government to rise above the fray and insulate itself from these pressures, as it is made up from the same people as make up all of the companies that jockey for position in the market.

Some might say that regulatory capture is the fault of the government – if the government didn’t exist, how could companies lobby it?  How could they get regulations passed that benefit them and harm their competitors?  The answer to this is very simple – if there is no government to stop a company growing, gaining power and monopolising, they can simply become the government.  Perhaps at first this might be done at a local level, where they acquire enough power to be able to dictate what can and cannot be done in that locality, at which point they have become a de-facto local government.  All they need to do is to continue to expand, and the sky’s the limit.

Effectively, there is always a government in one way or another.  A government is simply the power structure that places restrictions on what people can do:

• In dictatorships, the government is the dictator (and perhaps their generals).
• In representative democracies, the government is the group of people selected by the populace to govern.
• In anarchies, the government is whoever or whatever is the most powerful entity in a given situation, place and time.

If you reduce the size of government, and allow companies to grow larger and more consolidated, they become the power structure.  If this goes far enough, there is no reason that a company could not capture, subsume and replace the government entirely, making the CEO effectively the dictator, propped up by their “generals”, the key shareholders.  As long as the generals are kept happy, the dictator stays in power.

Going in the other direction, there are several governments that have reacted to the existential threat posed by companies in a free market by restricting the markets and companies heavily.  By crushing private enterprise out of existence, these governments may have avoided some of the excesses of free-market capitalism, but they have also forgone the huge benefits of the free market as an optimisation engine.  The government gaining control of a huge command economy is a hugely risky endeavour that leaves the country open to mismanagement and corruption.  A single failure can cause catastrophic economic collapse, as there is no backup – unlike a market system, the failure is not contained within a company that can go bankrupt without cascading further.

Regulation Within Reason

Ultimately, for all its difficulty, keeping the free market aligned with happiness and human flourishing is a job that only the government can do.  In my view this is therefore a key function of the government, requiring dedicated and diligent thought.  As such, contrary to the views of some politicians, regulations in themselves cannot be viewed as wholly good or bad – there are good regulations and bad regulations.

Some regulations are essential to keep the market free and avoid it from being misaligned, while others hinder the free-market optimisation engine with no discernible benefit.  Many of the benefits of the free market arise from situations close to perfect competition, in which there are lots of small companies competing against each other, driving the price down for the consumers of their goods.  It therefore stands to reason that most regulations that push the market towards this state are beneficial, and any that move it away from this state need careful consideration over whether their benefits outweigh the reduction in competition.

Often regulations are put in place that benefit incumbent companies or introduce barriers to entry for newcomers (e.g. state hairdresser licenses in the US).  Even regulations that at first glance seem costly to existing companies can be beneficial to them, by maintaining a bureaucracy that is difficult for new market entrants to navigate.  They will fight to preserve any barrier to entry that protects their turf.

An excellent case study for the risks of eschewing regulation entirely is the recent situation with Texas’ power grid.  Robustness was sacrificed for efficiency, by which I mean that actions that would have guarded against the problems being faced were not taken, as cost-saving was prioritised.  In a more competitive environment, the companies in question would have suffered for this lack of foresight, by losing out to their more prepared competition or new market entrants.  In Texas however, these companies were able to use their strong market positions to price gouge, passing on the cost of their shortsightedness to the customers.  Without a government willing to put in place regulations that would either enforce more competitive markets, or enforce robustness on the companies directly, the profit motive and the happiness of the people of Texas were not aligned.

The interesting problem is finding a system of government that is adequately resistant to capture, whilst also not interfering with the positive aspects of the free market too much.  A system of government that is adequately resistant to populism, whilst remaining democratic and representative of the people.  A system of government that is powerful enough to resolve the problems it needs to resolve, without being so powerful and overconfident that it tries to resolve problems it doesn’t need to.

If this sounds like a rephrasing of some of the thoughts in Meditations on Moloch, that is not a coincidence.  This balance between sacrificing too much to the optimisation engine, losing your values/happiness, and sacrificing too little, losing your freedom/life is very much the same theme that was addressed there.  I am however more optimistic.  It is not a battle that can ever be won with finality – it is an ongoing struggle requiring constant vigilance to ensure that our goals remain compatible with the reality in which we live.  That being said, humans are an intelligent species – as we come to better understand the rules of the game, we can learn where to exert pressure to subtly change the nature of the game itself.  Keeping the optimisation engines of both the market and evolution working towards our own ends as much as we can.